Implementation and Evaluation of multimodal input/output channels for task-based industrial robot programming

Industrial robots are used in many diferent felds of application: from huge welding production lines for automobiles to assembly tasks in Small and Medium-sized Enterprises (SMEs). Such robot systems require specially trained employees, who know how to program these robots and how to teach them new tasks using TeachPads and specifc programming languages like KRL (Kuka) or RAPID (ABB). Programming these robots is not very intuitive, and the programmer has to be a domain expert for e.g. welding and programming to know how the task is optimally executed. For SMEs such employees are not afordable, nor cost-efective. Therefore a new system is needed where domain experts from a specifc area, like welding or assembly, can easily program a robot without knowing anything about programming languages or how to use TeachPads. Such a system needs to be fexible to adapt to new tasks and functions. These requirements can be met by using a task based programming approach where the robot program is built up using a hierarchical structure of process, tasks and skills. It also needs to be intuitive so that domain experts don’t need much training time on handling the system. Intuitive interaction is achieved by using diferent input and output modalities like gesture input, speech input, or touch input which are suitable for the current task. This master thesis focuses on the implementation of a user interface (GUI) for task based industrial robot programming and evaluates diferent input modalities (gesture, speech, touch, pen input) for the interaction with the system. The evaluation is based on a user study conducted with 30 participants as a Wizard-Of-Oz experiment, where non expert users had to program assembly and welding tasks to an industrial robot, using the previously developed GUI and various input and output modalities. The fndings of the task analysis and user study are then used for creating a semantic description which will be used in the cognitive robotics-worker cell for automatically inferring required system components, and to provide the best suited input modality.